Vehicle image processing device, vehicle image processing method, program and storage medium

ABSTRACT

A vehicle image processing device includes: a plurality of buffers configured to accumulate pieces of image data input individually and sequentially from a plurality of cameras installed in a vehicle so as to associate the pieces of image data with the cameras; a processor configured to select the buffer based on the state information of the vehicle and acquire the piece of image data from the selected buffer so as to perform image processing thereon; a signal line for transferring the pieces of image data in the buffers to the processor; and a transfer controller configured to output the piece of image data in the buffer required from the processor to the signal line.

TECHNICAL FIELD

The present disclosure relates to ambient environment recognition whichis performed with a plurality of cameras installed in a vehicle.

BACKGROUND

In recent years, a large number of technologies have been developed inwhich a plurality of cameras installed in a vehicle are used so as toperform ambient environment recognition on the vehicle and in which thusa brake is automatically applied to the vehicle depending on thesituation, with the result that the driving assistance and the automateddriving of vehicles proceed. In the technologies described above, sincenot only the present vehicle but also people therearound, other vehiclesand the like are moved, with consideration given to feedback from theambient environment recognition to control on alarms, braking and thelike, images shot with the cameras need to be processed in real time.However, when a processor is provided in each of the cameras, the costof the vehicle is significantly increased.

In order to solve such a problem, Patent Document 1 discloses that in aconfiguration in which a plurality of stereo cameras are individuallyconnected to I/Os and in which a CPU (image processing IC) uses a RAM soas to perform, as necessary, image processing on image data shot, asituation judgment/image selection portion determines which one of theimages shot with the stereo cameras is output to an image processingportion in accordance with a support mode, and that thus the imageprocessing portion (a preprocessing portion and the subsequent portions)performs the image processing. In this way, it is possible to avoid theneed to use a computation device whose processing capability is high andthe like, and thus it is possible to reduce the cost.

Likewise, Patent Document 2 discloses that a CPU (computation portion)capable of performing front recognition processing for the imageinformation of a front camera input to an input portion and rearrecognition processing for the image information of a rear cameraselects and performs processing executed as the front recognitionprocessing and the rear recognition processing, respectively, based onthe state information of the present vehicle (information indicating aforward movement and a backward movement and a travelling state). Inthis way, as compared with a case where CPUs are separately provided forthe front recognition processing and the rear recognition processing, itis possible to simply its configuration.

Moreover, Patent Document 3 discloses that part of captured imagesobtained by imaging the ambient environment of a vehicle is selected asa target region according to conditions of the vehicle including thetravelling environment and the travelling state of the vehicle, thatimage processing is performed on the selected target region and thus ascompared with a case where image processing is performed on the entirecaptured images, a processing burden can be reduced.

CITATION LIST Patent Literature

Patent Document 1: JP2013-93013A

Patent Document 2: JP2018-95067A

Patent Document 3: JP2015-210584A

SUMMARY

However, in Patent Document 1, in order for the image selection to beperformed with the situation judgment/image selection portion, all theimage data generated by shooting with the individual stereo cameras istemporarily input to the CPU. Hence, the CPU needs to perform processingfor capturing all the image data in a stage preceding the imageselection described above, and an operating time allocated to the imageprocessing is reduced accordingly. Likewise, in Patent Document 2, theCPU receives inputs of all the image information of the front camera andthe rear camera, performs correction processing (distortion correction)on all the image information which is input and thereafter performs theselection described above. Hence, the CPU needs to perform, for example,processing for capturing all the image data and processing on imageinformation which is not selected, and thus an operating time allocatedto the image processing is reduced accordingly. As described above, whenthe amount of processing performed by a processor for the imageprocessing is increased, in order to process shot images in real time,the processor is required to have higher processing capability, with theresult that it is likely that the cost of the vehicle cannot besufficiently reduced.

In view of the foregoing situation, an object of at least one embodimentof the present invention is to provide a vehicle image processing devicewhich can more rapidly perform, while reducing the cost of a vehicle,image processing on images shot with a plurality of cameras.

(1) A vehicle image processing device according to at least oneembodiment of the present invention includes: a plurality of buffersconfigured to accumulate pieces of image data input individually andsequentially from a plurality of cameras installed in a vehicle so as toassociate the pieces of image data with the cameras; a processorconfigured to select the buffer based on the state information of thevehicle and acquire the piece of image data from the selected buffer soas to perform image processing thereon; a signal line for transferringthe pieces of image data in the buffers to the processor; and a transfercontroller configured to output the piece of image data in the bufferrequired from the processor to the signal line.

In the configuration of (1) described above, the processor for the imageprocessing does not perform any processing on the image data until theimage data is transferred with the transfer controller from the bufferin which the image data shot with each of the cameras is stored, andperforms the image processing on only the image data acquired by thetransfer. In this way, it is possible to reduce the burden of theprocessor, and thus it is possible to allocate a larger amount ofoperating time to the image processing. Moreover, for example, theprocessor determines, from the image data of the individual camerasaccumulated in the buffers, based on the state information of thevehicle such as the vehicle speed and shift position, the buffers whichneed to be set to the target for the image processing so as to narrowthe cameras set to the target for the image processing to at least partthereof, and thus it is possible to reduce the number of pieces of imagedata which needs to be processed per unit time.

As described above, the processor for the image processing sets theimage data of the cameras determined based on the state information ofthe vehicle to the target for the image processing, and does not performprocessing until the image data set to the target is acquired, with theresult that it is possible to more rapidly (more quickly) performenvironment recognition (the detection of a person and an article)around the vehicle through the cameras. Hence, it is possible to avoidthe provision of the processor for each of the cameras and the use of anexpensive processor having more excellent processing capability, andthus it is possible to reduce the cost of the vehicle.

(2) In some embodiments, in the configuration of (1) described above,the processor determines, based on the state information, at least onebuffer of the buffers as a selection target, and sets a selectionfrequency of the at least one buffer included in the selection targethigher than selection frequencies of the other buffers so as tosequentially select the at least one buffer included in the selectiontarget.

In the configuration of (2) described above, the processor determines,from the cameras, one or more cameras (one or more buffers) which areset to the target for the image processing, sequentially acquires theimage data from the one or more buffers corresponding to the one or morecameras determined so as to perform the image processing and therebyperforms the image processing on a predetermined number of pieces ofimage data per unit time. As described above, the cameras which are setto the target for the image processing are narrowed to at least part ofthe cameras, and thus it is possible to reduce the number of pieces ofimage data which need to be processed per unit time, with the resultthat it is possible to perform environment recognition (detection ofmovement of a person and an article) around the vehicle through thecameras in a satisfactory real-time manner.

(3) In some embodiments, in the configuration of (1) and (2) describedabove, the vehicle image processing device further includes: a captureunit having the buffers and the transfer controller provided therein; awork buffer configured to hold the piece of image data output by thetransfer controller to the signal line and subjected to the imageprocessing performed by the processor; and a processor unit having theprocessor and the work buffer provided therein, and the signal line isconfigured to be able to connect the buffers in the capture unit and thework buffer in the processor unit to each other.

In the configuration of (3) described above, the pieces of image data ofthe individual cameras managed with the buffers are passed through thesignal line for connecting the capture unit and the processor unit, aretransferred to the work buffer provided in the processor unit and aresubjected to the image processing in the processor. The management andthe transfer processing of the buffers in the capture unit is separatedwith the image processing in the processor unit in terms of hardware inthis way, and thus as described above, it is possible to performenvironment recognition (the detection of a person and an article)around the vehicle through the cameras in a satisfactory real-timemanner.

(4) In some embodiments, in the configuration of (1) to (3) describedabove, the state information includes a vehicle speed, and the processorperforms the selection of the buffer based on the comparison of thevehicle speed and threshold values.

In general, there is an upper limit on the number of pieces of imagedata on which a processor can perform image processing per unit time.Hence, when the pieces of image data of the cameras are sequentiallyprocessed, as the number of cameras (buffers) is increased, the numberof pieces of image data per camera subjected to image processing at aunit time is decreased. As this number is decreased, a longer delayoccurs in the image processing on the pieces of image data which aresequentially fed from each of the cameras, and thus a time lag in theenvironment recognition through the image processing of the cameras isincreased.

In the configuration of (4) described above, based on the comparison ofthe vehicle speed and the threshold values, the processor makes aselection such as by determining the buffers which are set to the targetfor the image processing. As the vehicle speed is increased, the timelag in the environment recognition through the image processing is morelikely to be fatal whereas as the vehicle speed is decreased, the timelag is more likely to serve as an allowable range. Hence, as the vehiclespeed is decreased, even when a larger number of cameras are set to thetarget for the image processing, it is possible to appropriately performdriving assistance for the vehicle. Therefore, the cameras which are setto the target for the image processing are selected by the relationshipwith the vehicle speed, and thus even the processor whose processingcapability is relatively low can appropriately perform the drivingassistance for the vehicle through the environment recognition using thecameras in a satisfactory real-time manner.

(5) In some embodiments, in the configuration of (4) described above,the cameras include one or more of the cameras for shooting thesurrounding of the vehicle, the threshold values include a firstthreshold value and when the vehicle speed is less than the firstthreshold value, the processor selects the buffer in which the piece ofimage data shot with the one or more of the cameras for shooting theentire surrounding of the vehicle is accumulated.

For example, when the vehicle is stopped before being moved or in astate (state where travelling is started) where the vehicle is moved ata very low speed due to a creep phenomenon or the like, the vehicle canbe moved to various positions depending on a steering angle. Hence, withthe near-field cameras which have a relatively wide angle of view so asto be able to appropriately shoot a near field or the like, it isnecessary to monitor the entire surrounding of the present vehicle (atleast the entire orientation in a horizontal direction).

In the configuration of (5) described above, when the state (state wheretravelling is started) as described above is detected with the firstthreshold value, the processor sets images shot with one or more of thecameras (such as one or more of the near-field cameras) necessary forobtaining images around the vehicle (the entire orientation) to thetarget for the image processing. In this way, it is possible toappropriately perform driving assistance corresponding to conditions ofthe vehicle through monitoring with the cameras in a satisfactoryreal-time manner.

For example, when the cameras such as a total of four cameras forshooting the front, the back, the left and the right of the vehicle areinstalled in order to obtain the entire surrounding, the number ofcameras set to the target for the image processing is more likely to beincreased. Hence, when the images of the cameras are sequentiallyswitched one by one, and then the image processing is performed, as thenumber of cameras is increased, the processing cycle of the image datain the individual cameras (individual buffers) is prolonged. However,since the vehicle is moved at a very low speed, as long as the imageprocessing is allowed (is not problematic) as the driving assistance forthe vehicle, the image processing can be performed on the images shotwith the individual cameras.

(6) In some embodiments, in the configuration of (4) and (5) describedabove, the cameras include a camera for shooting an area ahead of thevehicle and a camera for shooting an area behind the vehicle, the stateinformation further includes a shift position of a gear of the vehicle,the threshold values include the first threshold value and a secondthreshold value which is more than the first threshold value and whenthe vehicle speed is equal to or more than the first threshold value andless than the second threshold value, and the shift position indicates aforward movement, the processor selects only the buffer in which thepiece of image data shot with the camera for shooting the area ahead ofthe vehicle is accumulated whereas when the shift position indicates abackward movement, the processor selects only the buffer in which thepiece of image data shot with the camera for shooting the area behindthe vehicle is accumulated.

For example, in a state (low-speed travelling state) where the vehiclecan be rapidly moved in a lateral direction with respect to thedirection of travelling depending on the steering angle, it is necessaryto use the near-field cameras (described above) and the like so as toextensively monitor an area in the direction of travelling of thepresent vehicle.

In the configuration of (6) described above, when the state (low-speedtravelling state) as described above is detected with the firstthreshold value and the second threshold value, the processor setsimages shot with the cameras for shooting images in the direction oftravelling of the vehicle to the target for the image processing. Inthis way, it is possible to appropriately perform driving assistancecorresponding to conditions of the vehicle through monitoring with thecameras in a satisfactory real-time manner.

(7) In some embodiments, in the configuration of (4) to (6) describedabove, the cameras include a far-field camera which shoots a far-fieldarea in the direction of travelling of the vehicle and a near-fieldcamera which shoots a near-field area in the direction of travelling ofthe vehicle and which has a wider angle of view than the far-fieldcamera, the threshold values include the second threshold value and whenthe vehicle speed is equal to or more than the second threshold value,the processor selects the buffer in which the piece of image data shotwith the far-field camera is accumulated whereas when the vehicle speedis less than the second threshold value, the processor selects thebuffer in which the piece of image data shot with the near-field camerais accumulated.

In a travelling state where the vehicle speed is relatively so high thatthe vehicle cannot be rapidly turned in direction by steering, the timeallowed for feedback (for example, automatic braking in the vehicle or awarning for the driver) from the environment recognition (the detectionof a person and an article) through the image processing to the driverof the vehicle or the like is more shortened. Hence, it is necessary tomonitor a distant area from the vehicle is monitored so as to detect atarget (a person and an article) in a distant area in an earlier stage.

In the configuration of (7) described above, when the travelling stateas described above is detected with the second threshold value, theprocessor sets images shot with the far-field cameras for shooting adistant area in the direction of travelling of the vehicle to the targetfor the image processing. As described above, for example, images shotwith a smaller number of cameras such as one camera are set to thetarget for the image processing, and thus it is possible toappropriately perform driving assistance corresponding to conditions ofthe vehicle through monitoring with the cameras in a satisfactoryreal-time manner.

(8) In some embodiments, in the configuration of (4) to (7) describedabove, the threshold values include a switching start threshold valueand a switching completion threshold value in which a difference withthe switching start threshold value is a first value, the processor isconfigured to perform the image processing on a predetermined number ofpieces of the image data per unit time and when the vehicle speed isbetween the switching start threshold value and the switching completionthreshold value, the processor switches the buffers to be selected suchthat, as the vehicle speed approaches the switching completion thresholdvalue from the switching start threshold value, the predetermined numberof pieces of image data on which the image processing is performed perthe unit time before the vehicle speed reaches the switching startthreshold value is replaced with the predetermined number of pieces ofimage data after the vehicle speed reaches the switching completionthreshold value.

When the cameras (buffers) which are set to the target for the imageprocessing are exactly switched at a predetermined threshold value, forexample, if the vehicle speed obtained from the vehicle includes anerror, appropriate buffers corresponding to the vehicle speed are notselected around the threshold value, with the result that it is likelythat the appropriate environment recognition is not performed.

In the configuration of (8) described above, the cameras (buffers) whichare set to the target for the image processing of the processor aregradually switched from the preceding stage so as to be completelyswitched at the predetermined threshold values (such as the firstthreshold value and the second threshold value described above). In thisway, even when the vehicle speed obtained from the vehicle has an error,the ambient environment recognition on the vehicle can be appropriatelyperformed.

(9) In some embodiments, in the configuration of (8) described above,when the vehicle speed reaches the switching start threshold value, theprocessor first switches the camera corresponding to the buffer whichneeds to be selected when the vehicle speed reaches the switchingcompletion threshold value and the buffer corresponding to the camerawhich is least associated with a direction of shooting.

In the configuration of (9) described above, even when the vehicle speedobtained from the vehicle has an error, the ambient environmentrecognition on the vehicle can be appropriately performed.

(10) In some embodiments, in the configuration of (1) to (9) describedabove, the vehicle image processing device further includes: anacquisition portion acquiring characteristic information including aninstruction for the monitoring direction in a specific place, and theprocessor selects the buffers based on the position of travelling andthe characteristic information.

In the configuration of (10) described above, the processor sets theimage data of the camera for shooting the area in the monitoringdirection indicated by the characteristic information to the target forthe image processing. Although in the vehicle which travels in variousplaces, it may be necessary to monitor the specific direction in afocused manner such as a place in which accidents often occur, thecameras which are set to the target for the image processing areswitched based on the characteristic information, and thus it ispossible to perform the driving assistance which is safer.

(11) In some embodiments, in the configuration of (1) to (10) describedabove, the vehicle is an industrial vehicle, the state informationincludes the vehicle speed and the steering angle, the processordetermines, based on the state information, whether turning of thevehicle is rightward turning or leftward turning and when the vehicle isturned rightward, the processor selects the buffer in which the piece ofimage data obtained by shooting at least an area on a right side in aforward direction and an area on a left side in a backward directionwith respect to the direction of travelling is accumulated whereas whenthe vehicle is turned leftward, the processor selects the buffer inwhich the piece of image data obtained by shooting at least an area onthe left side in the forward direction and an area on the right side inthe backward direction with respect to the direction of travelling isaccumulated.

For example, since an industrial vehicle such as a forklift has a shortwheel base and a large steering angle, it is necessary to pay sufficientattention to the lateral direction as compared with the forwarddirection with respect to the direction of travelling when travelling isstarted in a state where a steering wheel is turned. Specifically, it isnecessary to simultaneously monitor not only the right side in theforward direction with respect to the direction of travelling but alsothe left side in the backward direction with respect thereto at the timeof rightward turning and not only the left side in the forward directionwith respect to the direction of travelling but also the right side inthe backward direction with respect thereto at the time of leftwardturning.

In the configuration of (11) described above, the processor determinesthe direction of turning based on the vehicle speed and the steeringangle, and selects, based on the result of the determination, thecameras which are set to the target for the image processing. Asdescribed above, according to the direction of turning (steering angle)of the vehicle, the image data of not only one side such as the rightside but also the opposite side (left side) is set to the target for theimage processing, and thus at the time of lateral turning, whileattention is being paid to the direction opposite to the direction oftravelling, for example, the detection of a person and an article isperformed, with the result that it is possible to perform the drivingassistance which is safer.

(12) In some embodiments, in the configuration of (11) described above,the cameras include a right-side shooting camera for shooting an area onthe right side of the vehicle and a left-side shooting camera forshooting an area on the left side thereof, and when the vehicle isturned rightward, the processor performs the image processing on theregion of part on a left side of the pieces of image data individuallyshot with the right-side shooting camera and the left-side shootingcamera whereas when the vehicle is turned leftward, the processorperforms the image processing on the region of part on a right side ofthe pieces of image data individually shot with the right-side shootingcamera and the left-side shooting camera.

According to the direction of turning (steering angle) of the vehicle,the image data of not only one side such as the right side but also theopposite side (left side) is set to the target for the image processing,and thus the number of cameras set to the target for the imageprocessing is increased, with the result that the frequency at which theimage processing is performed with the processor on the image dataaccumulated in each of the buffers (cameras) is decreased (the cycle isdecreased).

In the configuration of (12) described above, the processor performs theimage processing on the region of part of each piece of image dataacquired when the vehicle is turned. For example, according to thesteering angle, the camera on the right side at the time of rightwardturning processes the region (the left side of the image) of only thefront side of the vehicle, and the camera on the left side processes theregion (the left side of the image) of only the back side of the vehiclein a limited manner, that is, the image processing is performed on onlythe regions of parts, with the result that it is possible to reduce theprolongation of the acquisition cycle of the image data from each of thebuffers. In this way, even when the number of cameras is increased, itis possible to perform driving assistance which is safer in asatisfactory real-time manner.

(13) In some embodiments, in the configuration of (1) to (12) describedabove, the state information includes the steering angle and the shiftposition of the gear of the vehicle, and the processor performs, basedon the steering angle and the shift position the image processing on aregion of part included in the pieces of image data acquired from thebuffers.

In the configuration of (13) described above, based on the steeringangle and the shift position, part of each piece of image data isextracted so as to be set to the target for the image processing. Inthis way, in each piece of image data, the image processing on thepartial region which does not include the travelling route of thevehicle predicted based on the steering angle and the shift position canbe omitted. Hence, it is possible to reduce the burden of the imageprocessing on each piece of image data, and thus it is possible toincrease the number of pieces of image data on which the imageprocessing can be performed per unit time. Consequently, it is possibleto cope with a performance time which is required in a high vehiclespeed or the like.

(14) In some embodiments, in the configuration of (1) to (13) describedabove, the cameras include a first camera for shooting an area in afirst direction of the vehicle and a second camera for shooting an areain a direction different from the first direction, and the vehicle imageprocessing device further includes an adjustment portion adjusting,based on the brightness of the pieces of image data individually shotwith the first camera and the second camera, the shooting parameter foreach of the cameras.

In the configuration of (14) described above, the frontlight andbacklight of the cameras which are set to the target for the imageprocessing are detected such that the shooting parameters for thecameras are adjusted, and thus it is possible to minimize influencecaused by variations in brightness produced by sunlight, with the resultthat it is possible to perform stable environment recognition.

(15) A vehicle image processing method according to at least oneembodiment of the present invention is performed by a processor of acomputer, the computer includes: a plurality of buffers configured toaccumulate pieces of image data input individually and sequentially froma plurality of cameras installed in a vehicle so as to associate thepieces of image data with the cameras; the processor configured toselect the buffer based on the state information of the vehicleincluding a vehicle speed and acquire the piece of image data from theselected buffer so as to perform image processing thereon; a signal linefor transferring the pieces of image data in the buffers to theprocessor; and a transfer controller configured to output the piece ofimage data in the buffer required from the processor to the signal lineand the vehicle image processing method include a step of performing theselection of the buffer based on comparison of the vehicle speed andthreshold values.

In the configuration of (15) described above, the same effects as in (1)described above are achieved.

(16) A vehicle image processing program according to at least oneembodiment of the present invention instructs a processor of a computerto perform a step of performing the selection of a buffer based on thecomparison of a vehicle speed and threshold values, and the computerincludes: a plurality of buffers configured to accumulate pieces ofimage data input individually and sequentially from a plurality ofcameras installed in a vehicle so as to associate the pieces of imagedata with the cameras; the processor configured to select the bufferbased on state information of the vehicle including the vehicle speedand acquire the piece of image data from the selected buffer so as toperform image processing thereon; a signal line for transferring thepieces of image data in the buffers to the processor; and a transfercontroller configured to output the piece of image data in the bufferrequired from the processor to the signal line.

In the configuration of (16) described above, the same effects as in (1)described above are achieved.

(17) A storage medium according to at least one embodiment of thepresent invention is a computer-readable storage medium storing thevehicle image processing program discussed in (16) described above.

In the configuration of (17) described above, the vehicle imageprocessing program stored in the storage medium is executed, and thusthe same effects as in (1) described above are achieved.

According to at least one embodiment of the present invention, a vehicleimage processing device is provided which can more rapidly perform,while reducing the cost of a vehicle, image processing on images shotwith a plurality of cameras.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram schematically showing the hardware configuration ofa vehicle image processing device according to an embodiment of thepresent invention;

FIG. 2 is a diagram showing an example of installation of a plurality ofcameras in a vehicle in the embodiment of the present invention;

FIG. 3 is a diagram showing the functional blocks of the vehicle imageprocessing device according to the embodiment of the present invention;

FIG. 4 is a flowchart showing a method of determining the buffers of aselection target in the embodiment of the present invention, and thecameras are selected according to a vehicle speed;

FIG. 5 is a diagram showing an order in which the buffers (cameras) areselected when the vehicle speed in the embodiment of the presentinvention is less than a first threshold value (V<L1);

FIG. 6 is a diagram showing an order in which the buffers (cameras) areselected when the vehicle speed in the embodiment of the presentinvention is equal to or more than the first threshold value and lessthan a second threshold value (L1≤V<L2);

FIG. 7 is a diagram showing an order in which the buffers (cameras) areselected when the vehicle speed in the embodiment of the presentinvention is equal to or more than the second threshold value (L2≤V);

FIG. 8 is a diagram for illustrating the stepwise switching of thebuffers around the first threshold value when the vehicle speed in theembodiment of the present invention is increased;

FIG. 9 is a diagram for illustrating the stepwise switching of thebuffers around the second threshold value when the vehicle speed in theembodiment of the present invention is increased;

FIG. 10 is a diagram showing directions and regions which are monitoredwhen a forklift vehicle in the embodiment of the present invention isturned;

FIG. 11 is a flowchart showing a method of determining the buffers whenthe forklift vehicle in the embodiment of the present invention isturned;

FIG. 12 is a diagram showing a travelling route in image data which isset to the target for image processing in the embodiment of the presentinvention; and

FIG. 13 is a diagram for illustrating a case where the vehicle imageprocessing device according to the embodiment of the present inventiondetermines the backlit state of the vehicle.

DETAILED DESCRIPTION

Some embodiments of the present invention will be described below withreference to accompanying drawings. However, the dimensions, thematerials, the shapes, the relative arrangements and the like ofconstituent components which are described as embodiments or shown indrawings are not intended to limit the scope of the present inventionbut are simply examples of description.

For instance, an expression of relative or absolute arrangement such as“in a direction”, “along a direction”, “parallel”, “orthogonal”,“centered”, “concentric” and “coaxial” shall not be construed asindicating only the arrangement in a strict literal sense, but alsoincludes a state where the arrangement is relatively displaced by atolerance, or by an angle or a distance whereby it is possible toachieve the same function.

For instance, an expression of an equal state such as “same” “equal” and“uniform” shall not be construed as indicating only the state in whichthe feature is strictly equal, but also includes a state in which thereis a tolerance or a difference that can still achieve the same function.

Further, for instance, an expression of a shape such as a rectangularshape or a cylindrical shape shall not be construed as only thegeometrically strict shape, but also includes a shape with unevenness orchamfered corners within the range in which the same effect can beachieved.

On the other hand, an expression such as “comprise”, “include”, “have”,“contain” and “constitute” are not intended to be exclusive of othercomponents.

FIG. 1 is a diagram schematically showing the hardware configuration ofa vehicle image processing device 1 according to an embodiment of thepresent invention. FIG. 2 is a diagram showing an example ofinstallation of a plurality of cameras 8 in a vehicle 9 in theembodiment of the present invention. FIG. 3 is a diagram showing thefunctional blocks of the vehicle image processing device 1 according tothe embodiment of the present invention.

The vehicle image processing device 1 (hereinafter the image processingdevice 1 as necessary) is a device for performing image processing onimages (image data G) individually shot with a plurality of cameras 8installed (incorporated) in the vehicle 9 which is a general automobileor an industrial vehicle such as a forklift. As shown in FIG. 1, in theimage processing device 1, the cameras 8 are individually connected withseparate wires or the like, and the images shot with the individualcameras 8 are input to the image processing device 1. The number ofcameras 8 is arbitrary, and is preferably determined according to thepurpose thereof and the specifications (such as an angle of view and adistance capable of performing shooting) of the cameras. In theindividual cameras 8, images shot with the cameras 8 are continuouslyinput to the image processing device 1.

In the embodiment shown in FIGS. 1 and 2, each of the cameras 8generates a predetermined number of images per second. In the presentembodiment, as shown in FIG. 2, a total of six cameras 8 are installedin the vehicle 9, and the individual cameras are installed so as toshoot in the direction of circumference of the vehicle 9. Morespecifically, four out of the six cameras 8 are near-field cameras 81for near-field shooting, and the two cameras are far-field cameras 82for far-field shooting. The near-field camera 81 has a wider angle ofview (viewing angle) than the far-field cameras so as to be able toappropriately shoot a relatively near field. Conversely, the far-fieldcamera 82 has a narrower angle of view than the near-field camera 81 soas to be able to appropriately shoot a relatively far field. Hence, thefour near-field cameras 81 are separately installed in four places ofthe front, the back, the left and the right of the vehicle 9 so as tofully shoot the entire circumference of the vehicle 9 with the fournear-field cameras 81. The two far-field cameras 82 are separatelyinstalled in two places of the front (front where a driver's sheet ispresent) and the back (rear) of the vehicle 9 so as to shoot distantareas in forward and backward directions.

In order to process, in real time, the shot images which aresequentially input from the cameras 8 installed in the vehicle 9 asdescribed above, as shown in FIG. 1, the image processing device 1includes a plurality of buffers 2, a processor 3, a signal line 4 and atransfer controller 5.

The configurations thereof will be individually described.

The buffers 2 are storage portions which are configured so as toaccumulate pieces of image data G that are individually and sequentiallyinput from the cameras 8 installed in the vehicle 9 for each of thecameras 8. As shown in FIG. 1, a dedicated buffer 2 is provided for eachof the cameras 8, and thus the image data G is managed (stored) for eachof the cameras 8. In the embodiment shown in FIG. 1, each of the buffers2 is a queue (FIFO), and thus the pieces of image data G are taken outin an order in which the pieces of image data G are input to the buffers2. Each of the buffers 2 may have a ring buffer structure. Each of thebuffers 2 may be set to have such a capacity as to be able to store onlya small number of pieces of image data G, and a memory which has asmaller capacity is used such that the cost can be reduced. In theembodiment shown in FIG. 1, an A/D converter 22 converts the analoginformation of the shot image from each of the cameras 8 into digitalinformation, and thus the image data G is accumulated in each of thebuffers 2. The transfer controller 5 which will be described latermanages the individual buffers 2 so as to accumulate (store) the imagedata G in the individual buffers 2.

The processor 3 selects the buffer 2 among the buffers 2 described abovebased on, for example, the state information S of the vehicle 9 such asa vehicle speed V and a shift position Sp and acquires the image data Gfrom the selected buffer so as to perform the image processing.Specifically, each processor 3 is operated (such as the computation ofdata) according to the command of a program (image processing program)loaded on a main storage device so as to sequentially acquire the imagedata G from any one of the buffers 2 and to perform the image processing(such as the detection of a person and an article) on the image data Gwhich is acquired. As shown in FIG. 1, the number of processors 3 isless than the number of buffers 2 in order to reduce the cost of thevehicle 9. In the embodiment shown in FIG. 1, the processor 3 includedin the image processing device 1 has one CPU shown in the figure.

More specifically, the processor 3 is operated according to the imageprocessing program which has a function shown in FIG. 3 so as todetermine at least one of the buffers 2 as a selection target among thebuffers 2 based on the state information S of the vehicle 9, and setsthe selection frequencies of the buffers 2 included in the selectiontarget higher than the selection frequencies of the other buffers 2excluded in the selection target so as to sequentially select one ormore of the buffers 2 included in the selection target. Specifically,for example, by setting the selection frequencies of the buffers 2excluded in the selection target to a significantly low frequency suchas zero, they may be prevented from being selected by the processor 3.In the embodiment shown in FIG. 3, the image processing programinstructs a computer to realize: a selection target buffer determinationportion 71 which determines the buffers 2 of the selection target; animage data acquisition portion 72 which sequentially selects one or moreof the buffers 2 determined and included in the selection target andwhich acquires the image data G of the selected buffers 2; and an imageprocessing portion 73 which performs the image processing on theacquired image data G.

In general, since the number of pieces of image data G on which oneprocessor 3 can perform the image processing per unit time is limited,when the image data G from all the cameras 8 is monitored in real time,for example, a processor 3 which has excellent processing capabilityneeds to be provided or the processor 3 needs to be provided for each ofthe cameras 8. However, in general, priority (importance) is present inthe direction which needs to be monitored according to travellingconditions, and for example, the necessity to monitor the directionopposite to the direction of travelling is low at the time oftravelling. Hence, each time the travelling conditions of the vehicle 9are determined based on the state information S of the vehicle 9, thecameras 8 which need to be set to the target for the image processingare limited to at least part of the cameras 8 installed in the vehicle 9according to the details of driving assistance which is performed, withthe result that the processor 3 is made to perform the image processing.In this way, even when a plurality of buffers 2 are included in theselection target, since the number of buffers 2 which are set to thetarget for the image processing is minimized, the frequency at which theprocessor 3 acquires the image data G is increased for each of thebuffers 2 (the cycle in which the image processing is performed isshortened), with the result that environment recognition using thecameras 8 can be performed in real time.

The signal line 4 is a transfer path for transferring the image data Gin each of the buffers 2 described above to the processor 3. The signalline 4 may be configured so as to serially transfer the image data G ormay be configured so as to parallel transfer the image data G. Hence,the image data G in each of the buffers 2 is passed through the signalline 4 so as to be transferred to the processor 3.

The transfer controller 5 can communicate with the processor 3 for imageprocessing described above, manages the accumulation of the image data Gin the buffers 2 and outputs the image data G required from theprocessor 3 to the signal line 4. In other words, the transfercontroller 5 appropriately stores the image data G in each of thebuffers 2 while managing the communication with the cameras 8 and memoryaddresses so as to appropriately store the communicated image data G ineach of the buffers 2. As described above, the processor 3 determinesthe buffers 2 which are set to the selection target based on the stateinformation S of the vehicle 9, and each time the processor 3sequentially selects the buffer 2 therein, the image data G of theselected buffer 2 is acquired. Here, when the processor 3 specifies thebuffer 2 to be acquired to the transfer controller 5 so as to requirethe image data G, the transfer controller 5 outputs the image data G ofthe required buffer 2 to the signal line 4, and thus transfers the imagedata G to the processor 3. The processor 3 may perform the requirement(control signal) described above for the transfer controller 5 throughthe signal line 4 or may perform it through a control signal line whichis provided separately.

As described above, the transfer of the image data G to the processor 3is performed by the transfer controller 5, and thus the processor 3 canallocate an operating time to the image processing without performingthe transfer processing on the image data G. Hence, the processor 3 canallocate a larger proportion of the processing capability to the imageprocessing, and thus even when the processor 3 has relatively lowprocessing capability, the processor 3 can appropriately perform drivingassistance through the individual cameras 8 in real time.

In short, in the image processing device 1 configured as describedabove, since the transfer controller 5 performs the transfer processingon the image data G, the processor 3 for the image processing does notperform any processing on the image data G until the image data G istransferred with the transfer controller 5 from the buffer 2 in whichthe image data G shot with each of the cameras 8 is stored, and performsthe image processing on only the image data G acquired by the transfer.Moreover, for example, the processor 3 determines, from the image data Gof the individual cameras 8 accumulated in the buffers 2, based on thestate information S of the vehicle 9, the buffers 2 which need to be setto the target for the image processing so as to narrow the cameras 8 setto the target for the image processing to at least part of the cameras8, and thus the number of pieces of image data which needs to beprocessed per unit time is reduced.

As described above, the processor 3 sets the image data G of the cameras8 determined based on the state information S of the vehicle 9 to thetarget for the image processing, and does not perform processing untilthe image data G set to the target is acquired, with the result that itis possible to more rapidly perform environment recognition (thedetection of a person and an article) around the vehicle 9 through thecameras 8. Hence, it is possible to avoid the provision of the processor3 for each of the cameras 8 and the use of an expensive processor 3having more excellent processing capability, and thus it is possible toreduce the cost of the vehicle 9.

In some embodiments, as shown in FIG. 1, the image processing device 1may further include: a capture unit 11 (board) in which a plurality ofbuffers 2 described above and the transfer controller 5 are provided; animage processing buffer 6 (work buffer) which holds the image data Gwhich is output by the transfer controller 5 to the signal line 4 and onwhich the processor 3 performs the image processing; a processor unit 12(board) in which the processor 3 and the image processing buffer 6 areprovided. In this case, the signal line 4 described above can connectthe buffers 2 in the capture unit 11 and the image processing buffer 6in the processor unit 12 to each other. The capture unit 11 describedabove may be formed with a programable logic device such as an FPGA.

In other words, the image data G which is transferred with the transfercontroller 5 according to the requirement of the processor 3 is storedin the image processing buffer 6 provided in the processor unit 12.Then, the processor 3 performs the image processing on the image data Gstored in the image processing buffer 6. The image processing buffer 6of the embodiment shown in FIG. 1 is a queue (FIFO), and the processor 3performs the image processing on one or more pieces of image data Gstored in the image processing buffer 6 in an order in which the piecesof image data G are stored in the image processing buffer 6.

In the embodiment shown in FIG. 1, the image processing device 1acquires the state information S of the vehicle 9 described above from avehicle state monitoring portion 14 such as an ECU which is installed inthe vehicle 9. More specifically, the vehicle state monitoring portion14 collects, through a vehicle network 15 (such as a CAN) installed inthe vehicle 9, the state information S of the vehicle 9 including thevehicle speed V and the shift position Sp, and stores the stateinformation S in a buffer (state information buffer 62) for storing thestate information S provided in the processor unit 12. The collection ofthe state information S with the vehicle state monitoring portion 14 andthe storage of the state information S in the state information buffer62 are cyclically performed, and thus the latest state information S isstored in the state information buffer 62. Hence, the processor 3accesses the state information buffer 62 described above so as to beable to acquire the latest state information S of the vehicle 9, anddetermines, the buffers 2 which are set to the selection target based onthe acquired state information S.

Although in the embodiment shown in FIG. 1, the vehicle state monitoringportion 14 is provided in the other unit on the side of the vehicle(control unit 13), the control unit 13 and the processor unit 12 may beconnected with the vehicle network 15 or may be connected with separatewires.

In the configuration described above, the image data G of the individualcameras 8 managed with the buffers 2 is passed through the signal line 4for connecting the capture unit 11 and the processor unit 12, istransferred to the image processing buffer 6 provided in the processorunit 12 and is subjected to the image processing in the processor 3. Themanagement and the transfer processing of the buffers 2 in the captureunit 11 and the image processing in the processor unit 12 are separatedin terms of hardware in this way, and thus as described above, it ispossible to perform the environment recognition (detection of movementof a person and an article) around the vehicle 9 through the cameras 8in a satisfactory real-time manner.

Some embodiments on a method of determining the buffers 2 of theselection target described above will then be described with referenceto FIGS. 4 to 10. FIG. 4 is a flowchart showing the method ofdetermining the buffers 2 of the selection target in the embodiment ofthe present invention, and the cameras 8 are selected according to thevehicle speed V. FIG. 5 is a diagram showing an order in which thebuffers 2 (cameras 8) are selected when the vehicle speed V in theembodiment of the present invention is less than a first threshold valueL1 (V<L1). FIG. 6 is a diagram showing an order in which the buffers 2(cameras 8) are selected when the vehicle speed V in the embodiment ofthe present invention is equal to or more than the first threshold valueL1 and less than a second threshold value L2 (L1≤V<L2). FIG. 7 is adiagram showing an order in which the buffers 2 (cameras 8) are selectedwhen the vehicle speed V in the embodiment of the present invention isequal to or more than the second threshold value L2 (L2≤V). FIG. 8 is adiagram for illustrating the stepwise switching of the buffers aroundthe first threshold value L1 when the vehicle speed V in the embodimentof the present invention is increased. FIG. 9 is a diagram forillustrating the stepwise switching of the buffers around the secondthreshold value L2 when the vehicle speed V in the embodiment of thepresent invention is increased. FIG. 10 is a diagram showing directionsand regions which are monitored when a forklift vehicle in theembodiment of the present invention is turned. FIG. 11 is a flowchartshowing a method of determining the buffers 2 when the forklift vehiclein the embodiment of the present invention is turned.

In the following description, it is assumed that the processor 3 isoperated according to the command of the image processing programdescribed above so as to achieve realization. It is also assumed that inthe first threshold value L1, the second threshold value L2 and a thirdthreshold value L3, a relationship of L1<L2<L3 holds true.

In some embodiments, the state information S of the vehicle 9 includesthe vehicle speed V. As shown in FIG. 4, the processor 3 may perform theselection of the buffers 2 described above based on the comparison ofthe vehicle speed V and predetermined threshold values L. In theembodiment shown in FIGS. 1 to 4, the processor 3 determines one or moreof the buffers 2 which are set to the selection target described aboveaccording to the command of the image processing program for theselection target buffer determination portion 71 based on the comparisonof the vehicle speed V and a plurality of threshold values L (L1 to L3).Specifically, in step S1 of FIG. 4, the processor 3 (selection targetbuffer determination portion 71) checks whether or not a driver intendsto move the vehicle 9 by whether the shift position Sp is not parking.According to the commands of the selection target buffer determinationportion 71 and the image data acquisition portion 72, when the shiftposition Sp is not parking (P), in step S2 and the subsequent steps, theselection of the cameras 8 (buffers 2) corresponding to the vehiclespeed V is performed.

In general, there is an upper limit on the number of pieces of imagedata on which a processor can perform image processing per unit time.Hence, when the pieces of image data G of the cameras 8 are sequentiallyprocessed, as the number of cameras 8 (buffers 2) is increased, thenumber of pieces of image data on which the image processing isperformed per unit time in each of the cameras 8 is decreased. As thisnumber is decreased, a longer delay occurs in the image processing onthe pieces of image data G which are sequentially fed from each of thecameras 8, and thus a time lag in the environment recognition throughthe image processing of the cameras is increased. However, as thevehicle speed V is increased, the time lag in the environmentrecognition through the image processing is more likely to be fatalwhereas as the vehicle speed V is decreased, the time lag is more likelyto serve as an allowable range. In other words, as the vehicle speed Vis decreased, even when a larger number of cameras 8 are set to thetarget for the image processing, it is possible to appropriately performdriving assistance for the vehicle 9. Hence, in the present embodiment,the buffers 2 are switched according to the vehicle speed V, and thusthe image processing is performed on images shot with cameras 8 whichhave a higher priority according to the driving assistance among thecameras 8 installed in the vehicle 9.

Specifically, for example, when the vehicle 9 is stopped before beingmoved or in a state (state where travelling is started) where thevehicle 9 is moved at a very low speed due to a creep phenomenon or thelike, the vehicle 9 can be moved to various positions depending on asteering angle. Hence, it is considered that with the near-field cameras81 which have a relatively wide angle of view so as to be able toappropriately shoot a near field or the like, it is necessary to monitorthe entire surrounding of the present vehicle.

Hence, in some embodiments, as shown in FIG. 2, the cameras 8 includeone or more of the cameras 8 (in FIG. 2, the four near-field cameras 81)for shooting the surrounding of the vehicle 9. The threshold values Ldescribed above which are compared with the vehicle speed V include thefirst threshold value L1. In this case, as shown in FIG. 4, when thevehicle speed V is less than the first threshold value L1 (V<L1), theprocessor 3 may select the buffers 2 in which the image data G shot withthe one or more of the cameras 8 for shooting the entire surrounding ofthe vehicle is accumulated.

In the embodiment shown in FIG. 4, when V<L1 in step S2, in step S3, asshown in FIG. 5, ambient environment recognition is performed by theimage processing on the image data G of the near-field cameras 81. Inother words, the processor 3 sets, to the selection target, the fourbuffers 2 (2 a to 2 d) in which the image data G individually input fromthe four near-field cameras 81 is accumulated. Here, although the orderin which the buffers 2 of the selection target are selected isarbitrary, for example, the selection may be performed as shown in FIG.5. In the embodiment shown in FIG. 5, since on the image data G obtainedby shooting with the near-field cameras 81, the processor 3 performs theimage processing in the order of a first near-field camera 81 a (thefront of the vehicle 9), a second near-field camera 81 b (the right sideof the vehicle 9), a third near-field camera 81 c (the back of thevehicle 9) and a fourth near-field camera 81 d (the left side of thevehicle 9) (see steps S31 to S34 in FIG. 5), the pieces of image data Gare sequentially acquired while the buffers 2 are being selected in theorder of a first buffer 2 a, a second buffer 2 b, a third buffer 2 c, afourth buffer 2 d, the first buffer 2 a, . . .

As in the embodiment shown in FIG. 1, in order to obtain the entiresurrounding, the number of cameras set to the target for the imageprocessing is more likely to be increased. Hence, when the images of thecameras 8 are sequentially switched one by one, and then the imageprocessing is performed, as the number of cameras 8 is increased, theprocessing cycle of the image data G in the individual cameras 8(individual buffers 2) is prolonged. However, since the vehicle 9 ismoved at a very low speed, as long as the image processing is allowed(is not problematic) as the driving assistance for the vehicle 9, theimage processing can be performed on the images shot with the individualcameras 8.

It is considered that even when the vehicle speed V is equal to or morethan the first threshold value L1, for example, in a state (low-speedtravelling state) where the vehicle 9 can be rapidly moved in a lateraldirection with respect to the direction of travelling depending on thesteering angle, it is necessary to use the near-field cameras 81 and thelike so as to monitor a wide area in the direction of travelling of thepresent vehicle.

Hence, in some other embodiments, as shown in FIG. 2, the cameras 8include at least the cameras 8 (in FIG. 2, the two near-field cameras 81installed in the front and back of the vehicle 9) for individuallyshooting an area ahead of the vehicle 9 and an area behind the vehicle9. The state information S of the vehicle 9 further includes the shiftposition Sp of the gear of the vehicle 9. The threshold values Ldescribed above which are compared with the vehicle speed V include thefirst threshold value L1 and the second threshold value L2 which is morethan the first threshold value L1. In this case, when as shown in FIGS.4 and 6, the vehicle speed V is equal to or more than the firstthreshold value L1 and less than the second threshold value L2(L1≤V<L2), and the shift position Sp indicates a forward movement, theprocessor 3 may select the buffer 2 in which the image data G shot withthe camera 8 (in FIG. 2, the first near-field camera 81 a) for shootingthe area ahead of the vehicle 9 is accumulated whereas when the shiftposition Sp indicates a backward movement, the processor 3 may selectthe buffer 2 in which the image data G shot with the camera 8 (in FIG.2, the third near-field camera 81 c) for shooting the area behind thevehicle 9 is accumulated.

In the embodiment shown in FIG. 4, when in step S2 described above, arelationship of V<L1 does not hold true, and in the subsequent step S4,L1≤V<L2, in step S5, the ambient environment recognition which isdifferent from that in step S3 described above is performed by the imageprocessing on the image data G of the near-field cameras 81.Specifically, as shown in FIG. 6, in step S61, the processor 3(selection target buffer determination portion 71) checks the shiftposition Sp. Then, when the shift position Sp indicates the forwardmovement, in step S62, the first buffer 2 a in which the image data G ofthe first near-field camera 81 a for shooting a near-field area ahead ofthe vehicle 9 is accumulated is set to the selection target. Bycontrast, when the shift position Sp indicates the backward movement, instep S63, the third buffer 2 c in which the image data G of the thirdnear-field camera 81 c for shooting a near-field area behind the vehicle9 is accumulated is set to the selection target.

By contrast, in a travelling state where the vehicle speed is relativelyso high that the vehicle cannot be rapidly turned in direction bysteering, the time allowed for feedback (for example, automatic brakingin the vehicle 9 or a warning for the driver) from the environmentrecognition (the detection of a person and an article) through the imageprocessing to the driver of the vehicle 9 or the like is more shortened.Hence, it is considered that a distant area from the vehicle 9 ismonitored, and that thus it is necessary to detect a target (a personand an article) in a distant area in an earlier stage.

Hence, in some embodiments, as shown in FIG. 2, the cameras 8 include:the far-field cameras 82 for shooting a far-field area in the directionof travelling of the vehicle 9; and the near-field cameras 81 whichshoot the near-field area in the direction of travelling of the vehicle9 and which have a wider angle of view than the far-field cameras 82.The threshold values L described above which are compared with thevehicle speed V include the second threshold value L2. As shown in FIGS.4 and 7, when the vehicle speed V is equal to or more than the secondthreshold value L2, the processor 3 may select the buffers 2 (2 e and 2f in FIG. 1) in which the image data G shot with the far-field cameras82 is accumulated whereas when the vehicle speed V is less than thesecond threshold value L2, the processor 3 may select the buffers 2 (2 ato 2 d in FIG. 1) in which the image data G shot with the near-fieldcameras 81 is accumulated.

In the embodiment shown in FIG. 4, when in step S4 described above, arelationship of V<L2 does not hold true, and in the subsequent step S6,L2≤V<L3 (or L2≤V), in step S7, the ambient environment recognition isperformed by the image processing on the image data G of the far-fieldcameras 82. Specifically, as shown in FIG. 7, in step S71, the processor3 (selection target buffer determination portion 71) checks the shiftposition Sp. Then, when the shift position Sp indicates the forwardmovement, in step S72, the processor 3 (the same as described above)sets, to the selection target, the fifth buffer 2 e in which the imagedata G of the first far-field camera 82 a for shooting the far-fieldarea ahead of the vehicle 9 is accumulated. By contrast, when the shiftposition Sp indicates the backward movement, in step S73, the processor3 sets, to the selection target, the sixth buffer 2 f in which the imagedata G of the second far-field camera 82 b for shooting a far-field areabehind the vehicle 9 is accumulated.

In the configuration described above, based on the comparison of thevehicle speed V and the threshold values L, the processor 3 makes aselection such as by determining the buffers 2 which are set to thetarget for the image processing. As described above, the cameras 8(buffers 2) which are set to the target for the image processing areselected by the relationship with the vehicle speed V, and thus even theprocessor 3 whose processing capability is relatively low canappropriately perform the driving assistance for the vehicle through theenvironment recognition using the cameras 8 in a satisfactory real-timemanner.

In some embodiments, as shown in FIGS. 8 and 9, the threshold values Ldescribed above which are compared with the vehicle speed V may includea switching start threshold value Ls and a switching completionthreshold value Le in which a difference with the switching startthreshold value Ls is a first value W. In this case, when the vehiclespeed V is between the switching start threshold value Ls and theswitching completion threshold value Le, the processor 3 may switch thebuffers 2 to be selected such that, as the vehicle speed V approachesthe switching completion threshold value Le from the switching startthreshold value Ls, a predetermined number of pieces of image data G onwhich the image processing is performed per unit time before the vehiclespeed V reaches the switching start threshold value Ls is replaced withthe predetermined number of pieces of image data G after the vehiclespeed V reaches the switching completion threshold value Le. In otherwords, a ratio of the camera 8 (in FIG. 8 the first near-field camera 81a) whose priority (importance) is high is gradually increased accordingto the vehicle speed V.

When the cameras 8 (buffers 2) which are set to the target for the imageprocessing of the processor 3 are exactly switched at a predeterminedthreshold value L (the first threshold value L1 or the second thresholdvalue L2), for example, if the vehicle speed V obtained from the vehicle9 includes an error, appropriate buffers 2 corresponding to the vehiclespeed V are not selected around the threshold value L, with the resultthat it is likely that the appropriate environment recognition is notperformed. Hence, in the present embodiment, the buffers 2 which are setto the selection target are gradually switched.

Here, since the processor 3 performs the image processing on thepredetermined number of pieces of image data G per unit time, thepredetermined number of pieces of image data G are formed with imagedata G acquired from one or more of buffers 2 of the selection target.The switching start threshold value Ls is the threshold value L whichprovides an opportunity to switch part of the buffers 2 acquiring thepredetermined pieces of image data G to any one of the one or more ofbuffers 2 selected by the processor 3 when the vehicle speed V reachesthe switching completion threshold value Le. The switching completionthreshold value Le is, for example, the first threshold value L1 or thesecond threshold value L2 described above, and the switching startthreshold value Ls is a value obtained by increasing or decreasing theswitching completion threshold value Le only by the first value W.Specifically, for example, when the switching completion threshold valueLe is the first threshold value L1 described above, the switching startthreshold value Ls is Le+W (where Le+W<L2) or Le−W. For example, whenthe switching completion threshold value Le is the second thresholdvalue L2 described above, the switching start threshold value Ls is Le+Wor Le−W (where Le−W>L1). Between the switching start threshold value Lsand the switching completion threshold value Le, as shown in FIGS. 8 and9, one or more threshold values (stepwise threshold value Lm) mayfurther be provided so that the switching described above is performedmore stepwise.

In the embodiment shown in FIG. 8, the switching completion thresholdvalue Le is the first threshold value L1 described above, and theswitching start threshold value Ls is a value obtained by decreasing thefirst threshold value L1 only by the first value W (Ls=Le−W). In thiscase, in the embodiment shown in FIGS. 1 to 3, when the vehicle speed Vis less than the first threshold value L1 (Le), the processor 3sequentially selects the four buffers 2 (2 a to 2 d) corresponding tothe four near-field cameras 81 (see FIG. 5) whereas when the vehiclespeed V is equal to or more than the first threshold value L1, theprocessor 3 selects only the first buffer 2 a corresponding to the firstnear-field camera 81 a which shoots the area ahead of the vehicle 9 (seeFIG. 6).

Hence, in a case where the shift position Sp indicates the forwardmovement and the vehicle speed V is sequentially increased from zero,when the vehicle speed V is less than the switching start thresholdvalue Ls (V<Ls), the processor 3 (image data acquisition portion 72)sequentially selects the first buffer 2 a, the second buffer 2 b, thethird buffer 2 c and the fourth buffer 2 d, thereafter returns to thefirst buffer 2 a again and repeatedly makes the selection in the sameorder. Thereafter, when the vehicle speed V reaches the switching startthreshold value Ls, the processor 3 (the same as described above)replaces at least one of the second buffer 2 b, the third buffer 2 c andthe fourth buffer 2 d among the buffers 2 (2 a to 2 d) which have so farbeen included in the selection target with the first buffer 2 a.

Here, as shown in FIG. 8, when the vehicle speed V reaches the switchingstart threshold value Ls, the processor 3 may switch the camera 8 (inFIG. 8, the first near-field camera 81 a for shooting the area ahead)corresponding to the buffer 2 (in FIG. 8, only the first buffer 2 a)which needs to be selected when the vehicle speed V reaches theswitching completion threshold value Le and the buffer 2 (in FIG. 8, thethird buffer 2 c) corresponding to the camera 8 (in FIG. 8, the thirdnear-field camera 81 c for shooting the area behind) which is leastassociated with the direction of shooting.

In the embodiment shown in FIG. 8, the processor 3 switches the thirdbuffer 2 c corresponding to the third near-field camera 81 c forshooting the area behind the vehicle 9 to the first buffer 2 a. In otherwords, when the vehicle speed V reaches the switching completionthreshold value Le (first threshold value L1), since a state where theimage data G in the four directions of the front, the back, the left andthe right of the vehicle 9 has so far been set to the target for theimage processing is changed to a state where only the area ahead ismonitored, and thus the backward direction is opposite to the forwarddirection, the image data G in the backward direction is said to beleast important among the left, the right and the back of the vehicle 9.Hence, when the vehicle speed V reaches the switching start thresholdvalue Ls, the processor 3 switches the third buffer 2 c.

In the embodiment shown in FIG. 8, the stepwise threshold value Lm isprovided between the switching start threshold value Ls and theswitching completion threshold value Le, and when the vehicle speed V isincreased from the switching start threshold value Ls so as to reach thestepwise threshold value Lm (V=Lm), the second buffer 2 b correspondingto the second near-field camera 81 b for shooting an area on the rightof the vehicle 9 is switched to the first buffer 2 a. Then, when thevehicle speed V is further increased so as to reach the switchingcompletion threshold value Le (V=Le), the processor 3 switches thefourth buffer 2 d corresponding to the fourth near-field camera 81 d forshooting an area on the left of the vehicle 9 to the first buffer 2 a.In this way, at the first threshold value L1, the processor 3 sets thefirst buffer 2 a corresponding to the first near-field camera 81 a tothe selection target so as to perform the image processing. Thus, theratio of the camera 8 (in FIG. 8 the first near-field camera 81 a) whoseimportance is high is gradually increased according to the vehicle speedV.

On the other hand, in the embodiment shown in FIG. 9, the switchingcompletion threshold value Le is the second threshold value L2 describedabove, and the switching start threshold value Ls is a value obtained bydecreasing the second threshold value L2 only by the first value W(Ls=Le−W). In this case, in the embodiment shown in FIGS. 1 to 3, asdescribed above, when the vehicle speed V is equal to or more than thefirst threshold value L1 and less than the second threshold value L2,the processor 3 selects only the first buffer 2 a corresponding to thefirst near-field camera 81 a for shooting the area ahead of the vehicle9 (see FIG. 6) whereas when the vehicle speed V is equal to or more thanthe second threshold value L2, the processor 3 selects only the fifthbuffer 2 e corresponding to the first far-field camera 82 a for shootingthe area ahead of the vehicle 9 (see FIG. 7).

Hence, in a case where the shift position Sp indicates the forwardmovement and the vehicle speed V is sequentially increased from thefirst threshold value L1, when the vehicle speed V is less than theswitching start threshold value Ls (V<Ls), the processor 3 selects onlythe first buffer 2 a. Thereafter, when the vehicle speed V reaches theswitching start threshold value Ls, the processor 3 includes (adds) thefifth buffer 2 e in the current selection target, and decrease theselection frequency of the first buffer 2 a which is the current sectiontarget. In the embodiment shown in FIG. 9, among three continuous piecesof image data G on which the image processing is performed, the firstbuffer 2 a of the last piece of image data G is replaced with the fifthbuffer 2 e, and thus the selection frequency of the first buffer 2 a islowered. Furthermore, when the vehicle speed V is increased so as toreach the stepwise threshold value Lm, the last two pieces among thethree continuous pieces of image data G are replaced with the fifthbuffer 2 e. Furthermore, when the vehicle speed V is increased so as toreach the switching completion threshold value Le, only the fifth buffer2 e is selected. In this way, a ratio of the camera 8 (in FIG. 9 thefirst far-field camera 82 a) whose importance is high is graduallyincreased according to the vehicle speed V.

Although in the above discussion, the case where the vehicle speed V isincreased is described, the same is true for a case where when thevehicle speed V is decreased, the vehicle speed V is passed through theswitching start threshold value Ls, the stepwise threshold value Lm andthe switching completion threshold value Le in this order.

In the configuration described above, the cameras 8 (buffers 2) whichare set to the target for the image processing of the processor 3 aregradually switched from the preceding stage so as to be completelyswitched at the predetermined threshold values L (such as the firstthreshold value L1 and the second threshold value L2 described above).In this way, even when the vehicle speed V obtained from the vehicle 9has an error, the ambient environment recognition on the vehicle 9 canbe appropriately performed.

In some embodiments, the image processing device 1 may further include acharacteristic information acquisition portion 74 (acquisition portion)which acquires characteristic information including an instruction forthe monitoring direction in a specific place, and the processor 3selects the buffer 2 based on the characteristic information. In thevehicle 9 which travels in various places, a place (specific place) ispresent in which it is preferable to monitor the specific direction in afocused manner as the driving assistance, and for example, a place ispresent in which accidents where a person jumps out from the rightwarddirection often occur. Hence, the image processing device 1 acquires thecharacteristic information described above so as to include, in theselection target, the buffer 2 corresponding to the camera 8 forshooting an area in the monitoring direction indicated by thecharacteristic information. Here, the selection frequency of the buffer2 corresponding to the camera 8 for shooting the area in the monitoringdirection indicated by the characteristic information may also beincreased beyond those of the other buffers 2 included in the selectiontarget.

More specifically, the image processing device 1 (image processingprogram) may further include, as shown in FIG. 3, in addition to thecharacteristic information acquisition portion 74 described above, aposition acquisition portion 75 which acquires the position oftravelling of the vehicle 9. In the embodiment shown in FIG. 3, thecharacteristic information described above is made to correspond to thespecific place, and when the position of travelling of the vehicle 9reaches the specific place corresponding to the characteristicinformation, the buffer 2 corresponding to the camera 8 for shooting thearea in the monitoring direction indicated by the characteristicinformation is included in the selection target. The characteristicinformation acquisition portion 74 and the position acquisition portion75 may be realized by the operation of the processor 3 according to thecommand of the image processing program or may be realized by use ofhardware. In the embodiment shown in FIG. 3, the instruction for themonitoring direction is input to the selection target bufferdetermination portion 71, and thus the buffer 2 corresponding to thecamera 8 for shooting the area in the monitoring direction is includedin the selection target.

The position of travelling of the vehicle 9 may be acquired by aself-position recognition technology such as GPS (Global PositioningSystem) or SLAM (Simultaneous Localization and Mapping) or the positionof travelling (position information) may be acquired from the outside bycommunication with a beacon station installed in a road, communicationwith a base station in a mobile communication network or communicationwith the outside by use of a wireless communication technology such as anear-field communication using RFID (Radio Frequency ID entification).The specific place and the characteristic information are previouslydatabased, and the information thereof may be acquired by performingcommunication with the outside as necessary or the image processingdevice 1 may hold this database. Alternatively, in the image processingdevice 1, a device such as a beacon station may be installed near thespecific place, and the buffer 2 corresponding to the camera 8 whichshoots, when communication from the device such as a beacon station isreceived, based on the information of the monitoring direction includedin the characteristic information obtained by this communication, anarea in the direction thereof may be included in the selection target.

In the configuration described above, the processor 3 sets the imagedata G of the camera 8 for shooting the area in the monitoring directionindicated by the characteristic information to the target for the imageprocessing. Although in the vehicle 9 which travels in various places,it may be necessary to monitor the specific direction in a focusedmanner such as a place in which accidents often occur, the cameras 8which are set to the target for the image processing are switched basedon the characteristic information, and thus it is possible to performthe driving assistance which is safer.

In some embodiments, for example, when the vehicle 9 is an industrialvehicle, the state information S of the vehicle 9 includes the vehiclespeed V and a steering angle A, and the processor 3 may determinewhether the turning of the vehicle 9 is rightward turning or leftwardturning based on the state information S. Then, when the vehicle 9 isturned rightward, the processor 3 may select the buffer 2 in which theimage data G obtained by shooting at least an area on the right side inthe forward direction and an area on the left side in the backwarddirection with respect to the direction of travelling is accumulatedwhereas when the vehicle 9 is turned leftward, the processor 3 mayselect the buffer 2 in which the image data G obtained by shooting atleast an area on the left side in the forward direction and an area onthe right side in the backward direction with respect to the directionof travelling is accumulated.

As shown in FIG. 10, for example, since an industrial vehicle such as aforklift has a short wheel base and a large steering angle, it isnecessary to pay sufficient attention to the lateral direction ascompared with the forward direction with respect to the direction oftravelling when travelling is started in a state where a steering wheelis turned. Specifically, it is necessary to simultaneously monitor notonly the right side in the forward direction with respect to thedirection of travelling but also the left side in the backward directionwith respect thereto at the time of rightward turning and not only theleft side in the forward direction with respect to the direction oftravelling but also the right side in the backward direction withrespect thereto at the time of leftward turning.

However, when as described above, the image data G obtained by shootingan area on the side of the direction opposite to the direction oftravelling is also set to the target for the image processing, thenumber of cameras 8 on which the processor 3 performs the imageprocessing is simply increased, and thus the selection frequencies ofthe buffers 2 included in the selection target are decreased (the cycleof the selection is prolonged).

Hence, in some embodiments, as shown in FIGS. 10 and 11, the cameras 8include a right-side shooting camera (in FIG. 2, the second near-fieldcamera 81 b) for shooting the area on the right side of the vehicle 9and a left-side shooting camera (in FIG. 2, the fourth near-field camera81 d) for shooting the area on the left side thereof. The processor 3 isoperated according to the command of the image processing program forthe image processing portion 73, and thus at the time of rightwardturning, the image processing may be performed on the region (partialregion Gp, in FIG. 10, the left half of the image data G) of part of theleft side of the image data G individually shot with the right-sideshooting camera and the left-side shooting camera described abovewhereas at the time of leftward turning, the image processing may beperformed on the partial region Gp (in FIG. 10, the right half of theimage data G) of the right side of the image data G individually shotwith the right-side shooting camera and the left-side shooting cameradescribed above. In this way, it is possible to perform the drivingassistance which is safer in a satisfactory real-time manner.

FIG. 11 corresponds to transition from step S2 to step S3 in FIG. 4, andwhen in step S2 of FIG. 4, the vehicle speed V is less than the firstthreshold value L1 described above (V<L1), in step S21 of FIG. 11, thesteering angle A is checked, and when the steering angle A is equal toor less than a steering angle threshold value At (A≤At), the processproceeds to step S3 of FIG. 4 as already described. By contrast, when instep S21 of FIG. 11, the steering angle A is more than the steeringangle threshold value At (A>At), in step S22, the steering angle A ischecked. Then, when the steering angle A indicates rightward turning, instep S23, the image processing is performed on the region of the lefthalf of the image data G individually shot with the right-side shootingcamera and the left-side shooting camera described above. By contrast,when the steering angle A indicates leftward turning, in step S24, theimage processing is performed on the region of the right half of theimage data G individually shot with the right-side shooting camera andthe left-side shooting camera described above.

In the configuration described above, the processor 3 determines thedirection of turning based on the vehicle speed V and the steering angleA, and selects, based on the result of the determination, the cameras 8which are set to the target for the image processing. As describedabove, according to the direction of turning (steering angle) of thevehicle, the image data G of not only one side such as the right sidebut also the opposite side (left side) is set to the target for theimage processing, and thus at the time of lateral turning, whileattention is being paid to the direction opposite to the direction oftravelling as well, for example, the detection of a person and anarticle is performed, with the result that it is possible to perform thedriving assistance which is safer.

Another embodiment will then be described with reference to FIGS. 12 and13. FIG. 12 is a diagram showing a travelling route in the image data Gwhich is set to the target for the image processing in the embodiment ofthe present invention. FIG. 13 is a diagram for illustrating a casewhere the vehicle image processing device 1 according to the embodimentof the present invention determines the backlit state of the vehicle 9.

In some embodiments, the state information S of the vehicle 9 describedabove includes the steering angle A and the shift position Sp. As shownin FIG. 12, the processor 3 is operated according to the command of theimage processing program for the image processing portion 73 so as toperform the image processing on the region (partial region Gp) of partincluded in the image data G acquired from the buffers 2 based on thesteering angle A and the shift position Sp. Although in a camera imageshown in FIG. 12, a horizon h is shown in a distant area, and a road ris curved in the rightward direction, when the vehicle 9 travels alongthe road r, it is considered that the necessity to perform automaticdetection processing on the regions other than the travelling route isnot high. As described above, the image processing is performed on onlythe partial region Gp of the image data G, and thus it is possible todecrease the amount of information of each piece of image data G onwhich the image processing is performed, with the result that it ispossible to reduce the performance time of the image processing on onepiece of image data G.

For example, in some embodiments, as shown in FIG. 12, based on thesteering angle A and the shift position Sp, the route on which thevehicle 9 travels may be estimated, and based on the result of theestimation, the partial region Gp of the image data G may be determined.In the embodiment shown in FIG. 12, the processor 3 performs the imageprocessing on the estimated travelling route, and performs, for example,the automatic detection processing on only the estimated travellingroute. For example, when the threshold values L which are compared withthe vehicle speed V described above include the third threshold valueL3, and as shown in FIG. 4, the vehicle speed V is equal to or more thanthe third threshold value L3, the processor 3 may perform, based on theresult of prediction of the travelling route based on the steering angleA and the shift position Sp, the image processing on the partial regionGp included in the image data G acquired from the buffers 2. Althoughthe prediction of the travelling route may be performed by a knownmethod, the prediction of the travelling route may be performed withconsideration given to the size of the present vehicle, the position inwhich the camera 8 is attached, the angle (optical axis angle) at whichthe camera 8 is attached and the internal parameters of the camera 8. Inthis way, for example, when the vehicle speed V exceeds the secondthreshold value L2 and is further increased, the number of cameras 8which are set to the target for the image processing is narrowed, andmoreover, the region of each piece of image data G acquired on which theimage processing is performed is decreased, with the result that thenumber of pieces of image data G on which the processor 3 can performthe image processing per unit time is increased and that thus it ispossible to perform the environment recognition in real time.

In some other embodiments, for example, the regions of an upper portionand part of the left and right of the image data G which is determinedaccording to the vehicle speed V and the steering angle A are removedfrom the target of the image processing, and thus based on the vehiclespeed V, the steering angle A and the shift position Sp, the partialregion Gp of the image data G on which the image processing needs to beperformed may be determined. For example, in FIG. 12, it is found fromthe steering angle A that the vehicle 9 travels so as to turn to theright, and thus with consideration given to the vehicle speed V, regionsof an upper portion and a left side of the shot image may be removedfrom the target for the image processing.

In the configuration described above, based on the steering angle A andthe shift position Sp, part of each piece of image data G is extractedso as to be set to the target for the image processing. In this way, ineach piece of image data G, the image processing on the partial regionwhich does not include the travelling route of the vehicle predictedbased on the steering angle A and the shift position Sp can be omitted.Hence, it is possible to reduce the burden of the image processing oneach piece of image data G, and thus it is possible to increase thenumber of pieces of image data G on which the image processing can beperformed per unit time. Consequently, it is possible to cope with aperformance time which is required in a high vehicle speed V or thelike.

In some embodiments, as shown in FIG. 13, the cameras 8 include a firstcamera 8 a for shooting an area in one direction of the vehicle 9 and asecond camera 8 b for shooting an area in a direction different from theone direction. As shown in FIG. 3, the image processing device 1 mayfurther include an adjustment portion 76 which adjusts, based on thebrightness of the image data G individually shot with the first camera 8a and the second camera 8 b described above, shooting parameters foreach of the cameras 8. The processor 3 described above may be operatedaccording to the command of the image processing program such that theadjustment portion 76 is realized or the adjustment portion 76 may berealized by use of another type of hardware. In the embodiment shown inFIG. 13, the first camera 8 a is the first far-field camera 82 a, andthe second camera 8 b is the second far-field camera 82 b which shootsan area in a direction opposite to the one direction shot with the firstcamera 8 a. However, the present invention is not limited to the presentembodiment. In some other embodiments, the first camera 8 a and thesecond camera 8 b do not need to be the same type as long as thedirections of shooting are different between the first camera 8 a andthe second camera 8 b, and for example, the first camera 8 a is thefar-field camera 82 and the second camera 8 b is the near-field camera.

As described previously, since the entire surrounding of the vehicle 9is shot with the cameras 8, in an outdoor area, depending on apositional relationship with sunlight, some cameras 8 are backlit andsome camera 8 are frontlit, and when the shooting parameters for all thecameras are the same, it is likely that there is a camera which cannotobtain images of appropriate image quality (for example, brightness).Hence, in the present embodiment, for example, each time cameras 8 otherthan the cameras 8 which are set to the target for the image processingare temporarily included in the selection target, the brightness of theimage data G is evaluated such that an ambient environment on light andthe like is estimated and thus the shooting parameters for the cameras 8are automatically adjusted as necessary, with the result that in anyenvironment, the image data G of appropriate image quality can beobtained from the individual cameras 8.

Specifically, in the embodiment described above, when the vehicle speedV is equal to or more than the first threshold value L1, the environmentrecognition is performed with only the camera 8 (the first near-fieldcamera 81 a or the first far-field camera 82 a) for shooting the area inthe direction of travelling, and for example, this camera 8 is the firstcamera 8 a described above. Here, the processor 3 cyclically (forexample, once per 10 seconds) acquires the image data G shot with thecamera 8 (the third near-field camera 81 c or the second far-fieldcamera 82 b) for shooting the area in the direction opposite to thedirection of travelling or the like. For example, this camera 8 is thesecond camera 8 b described above. Then, the brightness of the imagedata G of each of the first camera 8 a and the second camera 8 b isevaluated.

Thereafter, for example, the average brightness (first averagebrightness I1) of a plurality of pieces of image data G formed with onlyimages shot with the first camera 8 a and the average brightness (secondaverage brightness I2) of a plurality of pieces of image data G formedwith images shot with the first camera 8 a and the second camera 8 b maybe individually evaluated. For example, among pieces of image data Gcontinuously acquired with time, the second average brightness I2 of aplurality of continuous pieces of image data G which includes the imagedata G obtained by shooting the area in the direction opposite to thedirection of travelling may be evaluated, and the first averagebrightness I1 of a plurality of continuous pieces of image data G whichdo not include the image data G in the opposite direction may beevaluated. Then, when I1<I2, it is determined that the camera 8 forshooting the area in the direction of travelling at that time is in thebacklit state, and thus the setting of the shooting parameters for thefirst camera 8 a is changed such that, for example, the image data Ggenerated by the cameras 8 in the backlit state is made bright and thatthus it is possible to perform appropriate shooting. By contrast, whenI1>I2, the setting is changed such that the brightness of the firstcamera 8 a is lowered.

In the embodiment shown in FIG. 13, the brightness of the image data Gof the first camera 8 a is relatively low due to backlight, and thebrightness of the image data G of the second camera 8 b is relativelyhigh due to frontlight. Hence, a relationship of I1<I2 holds true, andthus the adjustment portion 76 determines that the first camera 8 a forthe direction of travelling is backlit so as to change the setting ofthe shooting parameters and to thereby make the image data G of thefirst camera 8 a bright.

In the configuration described above, the frontlight and backlight ofthe cameras 8 which are set to the target for the image processing aredetected such that the shooting parameters for the cameras 8 areadjusted, and thus it is possible to minimize influence caused byvariations in brightness produced by sunlight, with the result that itis possible to perform stable environment recognition.

The present invention is not limited to the embodiments described above,and includes embodiments obtained by adding variations to theembodiments described above and embodiments obtained by combining theseembodiments.

The characteristic information acquisition portion 74 described above isan example of the acquisition portion.

The programs described above are software for instructing a computer torealize individual function portions which will be described later, andmay be stored in a computer readable storage medium.

1. A vehicle image processing device comprising: a plurality of buffersconfigured to accumulate pieces of image data input individually andsequentially from a plurality of cameras installed in a vehicle andrespectively shooting different surrounding regions of the vehicle so asto associate the pieces of image data with the cameras; a processorconfigured to select the buffer based on state information fordetermining traveling conditions of the vehicle and acquire the piece ofimage data from the selected buffer so as to perform image processingthereon; a signal line transferring the pieces of image data in thebuffers to the processor; and a transfer controller configured to outputthe piece of image data in the buffer required from the processor to thesignal line through selecting based on the state information.
 2. Thevehicle image processing device according to claim 1, wherein theprocessor determines, based on the state information, at least onebuffer of the buffers as a selection target, and sets a selectionfrequency of the at least one buffer included in the selection targethigher than selection frequencies of the other buffers so as tosequentially select the at least one buffer included in the selectiontarget.
 3. The vehicle image processing device according to claim 1,further comprising: a capture unit having the buffers and the transfercontroller provided therein; a work buffer configured to hold the pieceof image data output by the transfer controller to the signal line andsubjected to the image processing performed by the processor; and aprocessor unit having the processor and the work buffer providedtherein, wherein the signal line is configured to be able to connect thebuffers in the capture unit and the work buffer in the processor unit toeach other.
 4. The vehicle image processing device according to claim 1,wherein the state information includes a vehicle speed, and theprocessor performs the selection of the buffer based on comparison ofthe vehicle speed and threshold values.
 5. The vehicle image processingdevice according to claim 4, wherein the cameras include one or more ofthe cameras shooting a surrounding of the vehicle, the threshold valuesinclude a first threshold value and when the vehicle speed is less thanthe first threshold value, the processor selects the buffer in which thepiece of image data shot with the one or more of the cameras shootingthe entire surrounding of the vehicle is accumulated.
 6. The vehicleimage processing device according to claim 4, wherein the camerasinclude a camera shooting an area ahead of the vehicle and a camerashooting an area behind the vehicle, the state information furtherincludes a shift position of a gear of the vehicle, the threshold valuesinclude the first threshold value and a second threshold value which ismore than the first threshold value and when the vehicle speed is equalto or more than the first threshold value and less than the secondthreshold value, and the shift position indicates a forward movement,the processor selects only the buffer in which the piece of image datashot with the camera shooting the area ahead of the vehicle isaccumulated whereas when the shift position indicates a backwardmovement, the processor selects only the buffer in which the piece ofimage data shot with the camera shooting the area behind the vehicle isaccumulated.
 7. The vehicle image processing device according to claim4, wherein the cameras include a far-field camera which shoots afar-field area in a direction of travelling of the vehicle and anear-field camera which shoots a near-field area in the direction oftravelling of the vehicle and which has a wider angle of view than thefar-field camera, the threshold values include the second thresholdvalue and when the vehicle speed is equal to or more than the secondthreshold value, the processor selects the buffer in which the piece ofimage data shot with the far-field camera is accumulated whereas whenthe vehicle speed is less than the second threshold value, the processorselects the buffer in which the piece of image data shot with thenear-field camera is accumulated.
 8. The vehicle image processing deviceaccording to claim 4, wherein the threshold values include a switchingstart threshold value and a switching completion threshold value inwhich a difference with the switching start threshold value is a firstvalue, the processor is configured to perform the image processing on apredetermined number of pieces of the image data per unit time and whenthe vehicle speed is between the switching start threshold value and theswitching completion threshold value, the processor switches the buffersto be selected such that, as the vehicle speed approaches the switchingcompletion threshold value from the switching start threshold value, thepredetermined number of pieces of image data on which the imageprocessing is performed per the unit time before the vehicle speedreaches the switching start threshold value is replaced with thepredetermined number of pieces of image data after the vehicle speedreaches the switching completion threshold value.
 9. The vehicle imageprocessing device according to claim 8, wherein when the vehicle speedreaches the switching start threshold value, the processor firstswitches the camera corresponding to the buffer which needs to beselected when the vehicle speed reaches the switching completionthreshold value and the buffer corresponding to the camera which isleast associated with a direction of shooting.
 10. The vehicle imageprocessing device according to claim 1, further comprising: anacquisition portion acquiring characteristic information including aninstruction for a monitoring direction in a specific place, wherein theprocessor selects at least one of the buffers based on a position oftravelling and the characteristic information.
 11. The vehicle imageprocessing device according to claim 1, wherein the vehicle is anindustrial vehicle, the state information includes the vehicle speed anda steering angle, the processor determines, based on the stateinformation, whether turning of the vehicle is rightward turning orleftward turning and when the vehicle is turned rightward, the processorselects the buffer in which the piece of image data obtained by shootingat least an area on a right side in a forward direction and an area on aleft side in a backward direction with respect to the direction oftravelling is accumulated whereas when the vehicle is turned leftward,the processor selects the buffer in which the piece of image dataobtained by shooting at least an area on the left side in the forwarddirection and an area on the right side in the backward direction withrespect to the direction of travelling is accumulated.
 12. The vehicleimage processing device according to claim 11, wherein the camerasinclude a right-side shooting camera shooting an area on the right sideof the vehicle and a left-side shooting camera shooting an area on theleft side thereof, and when the vehicle is turned rightward, theprocessor performs the image processing on a region of part on a leftside of the pieces of image data individually shot with the right-sideshooting camera and the left-side shooting camera whereas when thevehicle is turned leftward, the processor performs the image processingon a region of part on a right side of the pieces of image dataindividually shot with the right-side shooting camera and the left-sideshooting camera.
 13. The vehicle image processing device according toclaim 1, wherein the state information includes the steering angle andthe shift position of the gear of the vehicle, and the processorestimates, based on the steering angle and the shift position, a routeon which the vehicle travels, and performs, based on a result of theestimation, the image processing on a region of part included in thepieces of image data acquired from the buffers.
 14. The vehicle imageprocessing device according to claim 1, wherein the cameras include afirst camera shooting an area in a first direction of the vehicle and asecond camera shooting an area in a direction different from the firstdirection, and the vehicle image processing device further comprises anadjustment portion adjusting, based on brightness of the pieces of imagedata individually shot with the first camera and the second camera, ashooting parameter for each of the cameras.
 15. A vehicle imageprocessing method performed by a processor of a computer, the computercomprising: a plurality of buffers configured to accumulate pieces ofimage data input individually and sequentially from a plurality ofcameras installed in a vehicle so as to associate the pieces of imagedata with the cameras; the processor configured to select the bufferbased on state information of the vehicle including a vehicle speed andacquire the piece of image data from the selected buffer so as toperform image processing thereon; a signal line transferring the piecesof image data in the buffers to the processor; and a transfer controllerconfigured to output the piece of image data in the buffer required fromthe processor to the signal line, wherein the vehicle image processingmethod comprises a step of performing the selection of the buffer basedon comparison of the vehicle speed and threshold values.
 16. Acomputer-readable storage medium storing a vehicle image processingprogram instructing a processor of a computer to perform a step ofperforming selection of a buffer based on comparison of a vehicle speedand threshold values, the computer comprising: a plurality of buffersconfigured to accumulate pieces of image data input individually andsequentially from a plurality of cameras installed in a vehicle so as toassociate the pieces of image data with the cameras; the processorconfigured to select the buffer based on state information of thevehicle including the vehicle speed and acquire the piece of image datafrom the selected buffer so as to perform image processing thereon; asignal line transferring the pieces of image data in the buffers to theprocessor; and a transfer controller configured to output the piece ofimage data in the buffer required from the processor to the signal line.